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PROJECT SUMMARY 

Grant Number 0EG2-6-000476-0514 
The Use of Follow-Up Studies in the Evaluation 
of Vocational Education 
Laure Sharp and Rebecca Krasnegor 
Bureau of Social Science Research, Inc. 

Objectives 

Three basic objectives were met by this study. 

1. To identify and describe follow-up studies pertaining to 
students of vocational education currently available or likely to become 

^available in the near future. Geographic, program, and time-span coverage 
were considered as part of this objective. 

2. To evaluate the extent to which this information is related 
to specific vocational education programs and thus serves as one 
measure of program effectiveness. 

3. To reveal information gaps and research needs, and establish 
a basis for making recommendations about appropriate techniques for 
meeting these needs. Emphasis here was both on methodological and 
substantive considerations. 

Procedures 

Procedures consisted essentially of the gathering of secondary 
source materials which were examined, and evaluated. Supplementary 
information (with respect to on-going and unpublished studies) was 
gathered through correspondence and personal interviews with the respective 
investigators. Data on on-going programs in the field of vocational 
education were sought primarily from program specialists in the U. S. 

Office of Education and private agencies active in this field. 

The report of this study consists of findings and recommendations 
with respect to the objectives as stated above and an annotated 



bibliography summarizing the pertinent publications and studies analyzed 
in connection with this report. 



Results and Recommendations 

Follow-up studies of vocational education program graduates were 
demonstrated to be useful tools in the evaluation of training and were 
recommended for future program assessment. Follow-up efforts were found 
to be common in the evaluation of vocational education programs in 
specific areas of the country to the exclusion of other areas and to 
have included some programs and omitted others. The most serious gap 
is the lack of follow-up information at the posthigh school level for 
those trained in technical institutes and junior colleges. Recommendations 
were made to provide systematic nationwide coverage for all levels of 
vocational training and for all vocational education programs. Although 
this should not be the only method of evaluation, it was felt that those 
who plan vocational education policies should have available to them 
data on the employment outcomes of those who have been trained. 

Perhaps the most important advance in vocational follow-up 
research is the current concern with the total system involved in training 
a person for an occupation--the training process, the characteristics of 
the graduate, the employment situation. The inclusion of the employment 
situation— short range and long-term opportunities for job seekers, 
actual skill requirements, evaluation of trained personnel by enployers, 
and predicted changes in employment situations and practices— makes 
possible a comprehensive evaluation of what happens to the individual 
who has been trained. 
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The most productive follow-up design for nationwide, comprehensive 
follow-up studies was found to be a combination of trend and cohort 



study. By matching vocational graduates in a given year with nonvocat ional 
graduates and conducting follow-up studies at different times would 
enable researchers to assess differences due to training, and would 
provide a clcervf picture of the effect of training over an extended 
period of time. At the same time, studies of new cohorts should be 
initiated at regular intervals and thus provide additional possible 
comparisons. At the same time, for such studies to have explanatory 
power, it will be necessary to plan for simultaneous study of other 



components of the “system’' within which training and placement take 
place: the school and the labor market. 

Furthermore, there is continuing need for intensive small scale 
studies of particular areas, programs, and factors. There is definite 
need to learn more about the role of specific institutional factors: type 

of school (comprehensive or vocational), size of school, role of teachers, 
curriculum experience and content, and guidance and placement services. 
These are only some of the areas where not enough systematic data are 
available. The need for study is equally strong in the area of labor 
market requirements, employer preferences, and behavior. 



THE USE OF FOLLOW-UP STUDIES IN THE EVALUATION 
OF VOCATIONAL EDUCATION 

K Definition of Follow-Up Studies 
Follow-up studies involve research designs which require a con- 
tact with individuals who have shared an experience in the past and whom 
the researcher desires to study or restudy. The usual goal of such 
studies is to arrive at some measure of the impact of the experience on 
the subsequent behavior or status of these individuals. In the area 
of vocational education the most widely accepted technique has been to 
evaluate training programs in terms of occupational outcome over a given 
period of time. The employment of a graduate in a job for which he 
received training is the accepted ultimate indicator of successful 
vocational training, although experts in the field recognize that many 
indicators other than training-related employment--for example enrollment 
in posthigh school training, or simple retention in high school through 
graduation — might be used to measure the “success" of vocational training.^ 
The usual technique for obtaining data concerning graduates is one or 
more follow-up contacts after training or occasionally, the collection 

of data on trainees through a particular phase of training, with subsequent 
follow-up. 

Studies mentioned in this report and listed in the annotated 
bibliography are considered to be representative of the use of the 

Report of the Panel of Consultants on Vocational Education, 
Education for a Changing World of Work, U. S. Department of Health, 
Education, and Welfare, Office of Education, Washington, D. C.: 1964, 

p. 89. 
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fol low-up methodology in evaluating vocational education programs. ^ We 
have attempted to cover the major prototypes and the most significant 
studies. However, there are many more, not included here, notably small 
studies of individual vocational systems and local programs whose findings 
usually have been disseminated on a local basL only. 

As will be seen, we have divided the studies reviewed into two 
major types, descriptive and explanatory studies. Like all typologies, 
this one is impure: some studies combine the two approaches. 

The difference between the two types is primarily a conceptual 
one. The basic methodology involved in conducting both types of follow-up 
studies is fundamentally the same, although different research designs 
are used. The studies require that either the subjects of the research-- 
graduates or students who had participated in training programs— be 
located and questioned about their current or past employment status, 
or that persons, such as teachers and employers, who have had contact 
with them, supply information about the actual subjects of the study. 

The extent to which the data collected through this follow-up method 
will yield meaningful findings depends on the quality of the data 
(appropriateness and randomness of samples, control for nonresponse bias, 
application of appropriate statistical and analytical techniques) as 
well as the quality of the design (for example, availability of control 
groups, provision for time series information, etc.). Thus both 
descriptive and explanatory studies can be good or bad, valid or invalid, 
and yield new insights or document the obvious. 

2 

The following vocational education programs were considered for 
this report: agriculture, home economics, trade and industry, technical, 

practical nursing and other health programs, distributive occupations. 
Office occupations and business occupations were omitted because of the 
recency of federal support of such programs. 
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Nevertheless it will become apparent to the reader that with a 
taw exceptions we tend to exhibit a bias in favor of the explanatory 
studies. Our technical reasons are set forth in later pages of this 
report and need not be anticipated here. But to a large extent, our 
bias is due to the fact that most descriptive studies are limited to 
data about the specific employment status of the training graduates at 
one point in time. The use of this methodology by researchers implies 
that of overriding importance in evaluating vocational training is the 
mere fact that the trainees are able to get jobs, without regard as to 
how well the individual performs in terms of job continuity, promotions, 
salary received, and amount of responsibility assigned to him. 

Descriptive studies need not be so narrow: colleges have often undertaken 

follow-up studies of their alumni which seek to measure the success of 
their programs by selecting a variety of criteria, including salary, 
job advancement, job responsibility, as well as activities unrelated to 
employment. The success of the alumni in all these spheres reflects 
successful education or training. In the field of vocational education, 
however, this broad perspective is seldom reflected in past descriptive 
studies which were examined for this report. 

The explanatory studies on the other hand offer hopeful signs 
that the narrow and self-conscious "placement" criteria are being 
replaced by more sophisticated evaluation devices, which take a wider view 
of employment outcomes, and attempt to relate specific training approaches 
to identifiable changes in the trainee's work life. Occupational results 
continue to be an important (or even the most important) indicator, but 
the explanatory study is usually more likely to provide knowledge about 
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the processes and situations which determine the relationship between 
training and occupation. In its most ambitious form— the study of the 
system within which all facets of vocational education interact— the 
explanatory study holds great promise as an evaluative tool. 

2. Descript ive Studies 

In general, this type of study describes the characteristics of 
individuals, groups, or situations by drawing inferences from data 
primarily with an informative rather than heuristic purpose. Among 
vocational education follow-up studies this type of study is found in 
statistical placement reports, one-time descriptive studies, trend 
studies, and longitudinal studies. 

The design of most descriptive studies involves a fairly 
uncomplicated method of following-up graduates after specific training 
and obtaining placement information. However, some descriptive studies 
rely not only on placement data as the major criterion by which to 
judge the success or failure of vocational training, but supplement 
placement data with information gathered from employers, school records, 
and various psychological tests. Trend and longitudinal studies involve 
more taxing data collection and analysis efforts than do placement 
reports and one-time descriptive studies. 

Placement Reports 

The design of a statistical placement report, as commonly used 
in the evaluation of vocational education, involves the location and 
contacting of vocational course graduates and the gathering of data 
which provide the researcher with information about the graduates' 
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placement status at the particular moment of contact. The findings are 
presented in the form of tabulations rather than detailed analysis of 
the data. A list of courses tabulated by number is the usual procedure. 
The North Atlantic Region placement reports demonstrate the relative 
ease with which data is obtained. Over the past ten years the proportion 
of course graduates for whom placement data was unobtainable averaged 
between five and ten per cent of all graduates. The reports show that 
the larger states in the region, particularly New York, have relatively 
the greatest difficulty in contacting graduates {13, 1^, 16). 

Studies using this design have often been conducted under the 
direction of an individual state as part of a report to the Federal 
Government. The North Atlantic Region reports are actually composite 
state reports. Vocational educators, both teachers and administrators, 
who may be untrained in research procedures contact the former students 
and collect the data. Theoretically, placement data is collected at a 
point in time after graduation, but the procedures are not always 
clearly spelled out and there is reason to believe that lack of control 
of untrained personnel may result in the confusion of employment plans 
obtained from the trainees before graduation with actual employment 
status six months after graduation. Among users of these data, there 
is some doubt about the reliability of this procedure. 

The format used to present placement information classified 
the graduates according to the following statuses: In jobs related to 

training, in jobs not related to training, continued in full-time school, 
entered armed forces, unemployed, and unaccounted for. The decision to 
classify a job as related or unrelated to training may be made on the 
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basis of the individual's self-report or through a comparison of job 
t teles and the particular program in which the graduate was enrolled. 
Obviously, there is considerably opportunity here for arbitrary decision 
making, -f^acluates in military services and those continuing their 
education are considered unavailable for employment. 

Any area, large or small, might be covered by such a study 
although practice has been for states or groups of states together to 
undertake this type of evaluation of their vocational education systems. 
Also, any vocational program or programs may be studied. 

As with all follow-up studies, the major difficulty is the 
location of the intended respondents. However, this problem is 
minimized here because contact usually occurs shortly after graduation. 
If the respondent cannot be located, there are persons still available 
in the area who know where he might be— parents, teachers, friends, 
etc. With a great deal of effort and money extended coverage is 
possible even many years after graduation. In the New Jersey study of 
graduates 91 per cent of the sample was reached ten years following 
high school graduation (3). In addition, the use of standardized 
placement status forms makes it relatively easy to conduct this type 
of study. 

Aside from the aforementioned lack of reliability inherent in 
the data collection, the primary inadequacy found in this study design 
lies in the fact that it is a statistical report of placement at only 
one time in the life of the individual rather than a study of career 
patterns. The basic assumption operative here is the idea that initial 
placement is crucial in determining long-term occupational choice and 
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career patterns. One major study of career patterns of one group of 
vocational students has in fact shown this to be the case. Eninger 
states that "if the graduate's first job is not in the trade studied or 
highly related, the chances are high that he will never enter the trade 
or a highly related trade. 8ut f if his first job is in the trade or 
a highly related trade, the odds are great that he will stay in the 
trade" (29'. This situation, of course, varies by occupation and may be 
less likely to persist in the future due to the more rapid rate of 
economic and technological change. 

One-Time Descriptive Studies 

A one-time descriptive study is more analytical in its focus. 

It illustrates a situation or describes individuals at one particular 
moment in time. In evaluating vocational education this method of study 
has involved an analysis of the graduate's employment and personal 
situation after he graduates (or drops out) in relation to his background, 
ability, and training received. The analysis, often extensive and 
detailed, is based on data gathered from the teacher, the employer, or 
the work supervisor in addition to or in place of information gathered 
from the graduate himself. For example, an evaluation of vocational 
education for practical nurses was undertaken in Michigan in 1957. 

Records of students and dropouts were studied, agencies evaluated the 
practical nurses they employed, and graduate nurses evaluated their own 
training. In another study, former merchandising students were 
described according to occupational status, age, training, place of 
residence, and job progress. These graduates also evaluated their own 
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training and made recommendations for improvement in the teaching and 
content of the courses. In a current project Mallinson is comparing 
the characteristics of persons of posthigh school age who are enrolled 
in vocational or technical school, are employed, or are looking for work, 
with the characteristics of their former classmates who have enrolled 
in higher education.. All of these former students had been studied and 
tested while in school (15, 24, 38). 

This type of descriptive study has often been limited in scope, 
concerned with a particular type of vocational education program of a 
particular school. Illustrative is a follow-up study of thirty-seven 
students of one high school distributive education program in Des 
Moines (8). However, there is no reason why such a study could not 
be accomplished on a wider scale with extended relevance for other 
vocational education systems. The most important weakness, as in the 
statistical placement report, is the reliance on information obtained 
at one particular mo.nent in time from which to draw conclusions for the 
past and future. However, this limitation is somewhat lessened by 
conducting the follow-up at an extended time after training, such as 
ten years after graduation from the course. Coe and Zanzalari conducted 
such a study in 1963 1 953 class graduates of technical training in 

three vocational high schools in New Jersey. Occupational placement 
data for each year during the ten year period of the study were obtained 
from 91 per cent of the graduates ( 3 ). The most comprehensive study 
along these lines was done by Little on all Wisconsin high school 
graduates (including those of vocational schools) 8 years after graduation. 
However, this study because of its analytical complexity, has been 
classified under "explanatory studies" below. 
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Trend Studies and Longitudinal Studies 

The trend study elaborates on the one-time descriptive study 
design by considering at least two groups or cohorts who had completed 
vocational training at different times. This allows for a variety of 
comparisons and provides insights about ongoing processes and trends, 
with respect to the relation between training and employment. This 
design was used in the AIR (Eninger) study of Trade and Industry 
graduates of three classes: 1953, 1958, and 1962. (The study is 

mentioned here because it presides the best illustration of the trend 
design. It is actually a hybrid between a descriptive and explanatory 
study, and belongs more properly into the latter category, where it is 
classified in the appended bibliography.) The three groups were studied 
in 1963, 10, 5 and 1 years after completing vocational training. These 
vocational course graduates were also compared with academic course 
graduates who attended the same comprehensive high school attended by 
half the vocational course graduates during the same class years. 

A longitudinal study, similar to the one-time descriptive study, 
but based on data collected from the same sample of individuals at several 
different points in time obtains a picture of career development rather 
than placement alone. In studies of the effectiveness of vocational 
education this has taken the form of before and after studies in which 
comparisons are made, in terms of employment, of career aspirations 
before graduation and actual goals achieved after graduation. Further- 
more, research has involved contacting individuals at specific time 
intervals, such as while in high school, shortly after high school 
graduation, and two years after graduation. By far the most ambitious 
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vocat ional education follow-up of this type involves six questionnaires 
administered over a twenty year period for the purpose of establishing 
a description of long-term career patterns of practical nurses (12). 
Obvious difficulties are involved in following respondents over a long 
period of time after the initial contact, but this design does give a 
more valuable and accurate picture of the success or failure of vocational 
training. Occasionally an attempt is made to compare trend and longitu- 
dinal data, which at least in theory, would be the most productive 
device, since the interaction of "career effects" and "trend effects" 
could be analyzed. To date, this rather sophisticated research approach 
has not been used. An attempt in this direction can be found in the 
study by Haines and Ozzelo of high school graduates of cooperative 
training in which they dealt with two cohorts; they followed up l 963 
graduates ten months after graduation (in 1964) and a sample of 1962 
graduates previously studied in 1 963 (29, 6). 

3. Explanatory Studies 

The explanatory study attempts to explain a situation, test a 
theory, or draw inferences of causality. There are two types, 
diagnostic, which search for possible causes in evaluating a situation, 
and experimental, which try to create new situations by manipulating the 
environment and introducing experimental factors to gain an understanding 
of the actual operation of a system. In these terms the experimental 
design tries to use the traditional scientific experimental method 
modified for the study of human social problems. Coster used an 
experimental design to assess the factors in vocational agricultural 
education which prepare students for agricultural occupations other 
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than farming. He did this by comparing three groups who received three 
different types of vocational training— work experience, classroom study, 
or part-time work— part- t i me study. In a similar study, Twyman is 
studying the effects of three different types of education— vocational , 
academic, or vocational and academic combined— on the career outcomes 
of high school dropouts who were reenrolled in school and assigned to 
one of three groups. These studies manipulate the training situation 
and then draw conclusions from the differing results of the respective 
training methods (28, k\). 

In a more unorthodox experiment Kaufman uses different types of 
‘•opportunities' 1 for high school dropouts to assess the economic and 
psychological effects of the differential situational settings. The 
"opportunities" involved are the opportunity to earn a high school 
diploma or to receive training for a specific occupation (33). 

Diagnostic studies are far mere common. The diagnostic design 
tries to measure various dimensions and aspects of a problem to broaden 
the basis for understanding. Characteristic of the use of this design 
is a series of studies being conducted by Ulman in California concerned 
with the relationship of the labor market to vocational education needs. 
Data gathering for this study involves surveying employers, union 
officials, employment and placement service personnel, as well as high 
school dropouts and graduates (*>2). Other diagnostic studies of vocational 
education students have used the explanatory methodology to study the 
part played by vocational education in the working lives of graduate 
trainees and the effectiveness of vocational education. In such studies 
the effect of vocational education on the lives and employment career 
patterns of the individuals trained is perhaps best exhibited. 

o 
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Data for these studies is collected from all or any of the 
sources which are connected with vocational education although custom 
has been to go directly to the individual concerned. Instruments 
commonly used to gather data include psychological tests, school records, 
and check lists of activities. Analysts is extensive and deeply 
involved in psychological, sociological, and economic dimensions. 

Several such studies are currently in progress, but none have been 
completed, although partial results are becoming available (34, 40) . 

Another approach to the diagnostic follow-up study of vocational 
students is primarily comparative, and involves groups who have received 
different types of vocational education. (In effect, to some extent 
this is an ex post facto "experimental" design, as described earlier.) 

Its purpose is to find explanations for differing employment histories 
in differing characteristics of groups, or in differing qualities and 
types of vocational training. Types of comparisons have involved 
differences of race, academic versus vocational training, completion 
and noncompletion of training, and on-the-job training versus 
institutional training. For instance, McKechnie makes comparisons 
between those who completed training and those who dropped out in terms 
of employment results and Kahn and Bachman plan to compare the effects 
of three patterns of education and socialization — high school, employment, 
and unemployment — among individuals who, when first contacted for study, 
are high school sophomores. Plans have been made to contact these 
students at eighteen month intervals over a three year period during 
which time some students will have experienced employment and unemployment 
(32, 39). 
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V/ith the development of sophisticated computer techniques and 
programs — enabling researchers to handle simultaneously a very large 
number of variables — it is possible, at least in theory, to design 
diagnostic studies which encompass the total vocation educational system. 
Project Talent with its design for detailed analysis of the American 
high school and high school students over an extended time span might 
have done this type of analysis easily if a larger number of vocational 
schools had been included in the sample. However, thirty- five vocational 
schools in a national sample of 17>878 high schoois did not yield 
sufficient data for comparisons between vocational and nonvocation 
school systems and students (30). 

Eninger’s study, . however, may eventually yield such an analysis 
for Trade and Industry graduates. This entire study (to date only 
partially analyzed) sought to describe and analyze the process and 
product of vocational education and rests on the following equation: 
vocational experience is the product of the interaction of the vocational 
school system, the vocational human (the student) and regional vocational 
characteristics (the job opportunity structure). 

These newer, diagnostic studies represent a radical change in 
the evaluation of vocational education, which up to this time has relied 
on placement in the field of training as the preferred method of 
evaluation. However, while each study has considerable methodological 
and substantive merit in the aggregate they are too limited in geographic 
and program coverage to supply a basis for the measurement of the 
results of vocational education efforts for the country as a whole. 




4. Areas Covered by Past and Current Research 

In the past, certain areas of the country and certain states 
have undertaken follow-up evaluations more often than others* The 
Northeast has been covered annually by the placement reports of thirteen 
states of the North Atlantic Region extending from Maine to West Virginia. 
Several midwestern and western states — Michigan, Wisconsin, Iowa, 

Illinois, California, Nebraska, and Oregon— have either conducted 
studies of their systems of vocational training or have been the 
location for follow-up studies within their boundaries. Except for 
occasional small studies the South has been virtually neglected by 
follow-up evaluation. This may not be altogether accidental. Vocational 
training for Negroes in southern states has been operating under 
restrictions imposed by employment discrimination patterns: Negroes 

were not offered training opportunities in fields where they were not 
traditionally employed (48) . 

The current emphasis appears to be directed toward national 
studies, such as the American Institute for Research study or the study 
of nursing education, which view one type of vocational program in all 
parts of the country (12, 29). This emphasis does not imply that 
individual vocational systems or particular areas within states are not 
being studies (28, 35, 36, 40). 

Time-span coverage in the past has concentrated on placement at 
one time in the job history of the individual soon after training has 
ended. However, there appears to be a trend to an increasing number 
of longitudinal or cohort-trend studies, ranging from before and after 
graduation studies, to the comparisons of different types of students 
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over time, or to panels studies, with recanvassing of the same subjects 
for many years. Such studies give a more continuous picture of employ- 
ment history of the individuals studied and the long-term effects of 
training than does a one time census type data collection but they are 
costly and difficult, largely because of attrition problems. 

Every major area of vocational education has been the subject 
of follow-up research at some time but not all types of programs have 
been studied equally in depth, at all levels of training. Most past 
and current research involves vocational education at the high school 
level — Eninger studied graduates of high school trade and industrial 
programs, Cook plans to study the relationship between high school 
training and entry jobs for office and retail trade occupations. 
Corazzini plans to follow technical students through high school and 
subsenuent placement, Levinson has studied a cross-section of vocational 
students after graduation from high school and the comprehensive studies 
by Sewell and Kaufman also deal with cross-sections of high school 
populations (29, 26, 27, 33, 35, 40). 

At the post secondary level there have been very few studies. 

The U. S. Office of Education has conducted brief follow-up studies of 
home economics teacher education; Schmitt and Tyler studied practical 
nurses in Michigan in 1957; McKechnie is studying students enrolled in 
night courses; and Mallinson is directing a study of students of post- 
secondary vocational education (21, 22, 15, 39, 38). At this level 
research coverage is particularly weak. Vie know practically nothing 
about the students or graduates of postsecondary or supplementary 
vocational education. A major gap in this area is lack of knowledge of 
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the junior college student enrolled in occupational training. In general, 
when follow-up studies of junior college students have been undertaken, 
concern has been with transfer to higher educational institutions rather 
than occupational outcome related to training even though many students 
are enrolled in the occupational rather than academic curriculum. The 
American Association of Junior Colleges has emphasized the great need to 
evaluate the vocational programs in junior colleges through follow-up 
of former students and urges junior colleges to undertake studies of their 
own graduates. The Association has tried to encourage such studies by 
publishing a guide to conducting follow-up studies which includes 
examples of questionnaires used by particular schools (45). 

Beyond studies of junior college students, very little research 
of any kind has been conducted which concerned itself with adults who 
have been enrolled in vocational education programs. Adult extension 
programs or part-time vocational training are of major importance in the 
area of vocational education both because of the number of individuals 
involved (in 1963, 45 per cent of the total enrollment in vocational 
education programs were adults enrolled in extension programs) and 
the relatively large proportion of federal funds (in 1964, 20% of the 
federal expenditures were for adult programs) allocated for adult 
vocational training. No doubt, this neglect of adult students in 
follow-up studies is primarily a result of the difficulty of locating 
the scattered population of adult students enrolled in part-time 
courses. Furthermore, in many instances, no systematic records are 
kept for this group of students. Yet, follow-up studies concerned with 
these persons are vitally necessary if vocational education is to be 
evaluated as a total system. We should know who these people are, why 
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they are taking these training courses, and what relationship this 
training has to their employment life. 

Program coverage has not been even. Most research has involved 
trades and industry, distributive education, or agricultural programs. 

Home economics and health occupations have not been studied at the high 
school level, but have been the subject of research at the posthigh 
school level where most other programs have been neglected. Although 
work is still being done on particular programs, such as the study of 
training students for nonfarm agricultural jobs by Coster, many current 
studies tend to consider jointly all vocational offerings or juxtaposing 
graduates from vocational and academic or general programs. For instance, 
Kaufman and Shaefer are studying 5,000 vocational, academic, and general 
curriculum graduates with a view to preparation for employment and 
Loomis is studying the relationship of vocational education training 
to specific requirements necessary for employment (28, 3^, 37). 

Rel iance on the use of crude placement data as the major 
criterion in assessing vocational training appears to be lessening as 
evidenced by new research planned to examine the economic situation of 
the area in which training is being evaluated. Furthermore, the 
researchers feel that it is not enough to know that an individual is 
employed in a field related to his high school training, although 
this is an important point. They want to know if the individual 
possessed the requisite skills for the job and if he is actually using 
his training in some way. Ulman plans to study all areas of the labor 
market in their relationships to training individuals for jobs and 
Cook will assess the employment requirements and the training received 
by recent graduates (42, 26). 
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A1 though many of the studies recently completed or now under way 
make substantive contributions to the problem of evaluating the occupational 
results of vocational training programs, there remain sizable gaps. The 
need for comprehensive geographic coverage is imperative, especially 
in order to obtain some baseline data on the results of vocational 
training offered to the large number of Negro youths who are educated in 
southern states. Inseparable from follow-up data on graduates are inputs 
about institutional factors of the vocational education system — the 
selection processes of the training course, the program retention of 
students, the role of teachers, and the content of the programs. For 
the graduates themselves, we need to know more about long-term career 

patterns, as well as about the students' attitudes toward employment 
and training. Neither has there been sufficient evaluation of outcome 
in relation to type of school — comprehensive or vocational — nor an 
assessment of the new trend in area technical schools. 

Although a few of the studies reviewed here deal with this 
issue, not enough information is as yet available about the nature of 
optimal preparation for the various types of jobs for which vocational 
students are trained. Broad and general job training, on-the-job work 
experience, very speci f fc^tra ining in a specific job-skill area, 
training in "job behavior and human relations," and combinations of 
these — which is preferred by students or employers and thought 

appropriate for successful long-term employment? A recent study 
by the U. S. Department of Labor throws some lignt on the way in which 
persons now in the labor force acquired their training (49), Current 
emphasis on specific skill training as preparation for a life-time 
occupation must be constantly evaluated in the light of trends in job 
o 
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innovation and obsolescence lest a situation result in which individuals 
trained for skilled jobs find their training outdated.^ 

Perhaps the most important advance in vocational education 
follow-up research is the concern with the total system involved in 
training a person for an occupat ion--the training process, the 

♦Or 

characteristics of the graduate, the employment situation. It is the 
inclusion of the employment situation — short range and long-term 
opportunities for job seekers, actual skill requirements, evaluation of 
trained personnel by employers, and predicted changes in employment 
situations and pract ices--which makes possible a comprehensive evaluation 
of what happens to the individual who has been trained. Until recently, 
this perspective was not incorporated in vocational education follow-up 
research. 



5» Long-Range Recommendations 

Follow-up studies of vocational education program graduates have 
been demonstrated to be useful tools In the evaluation of training and 
should be available as a regular input for future program assessment. 
Although this should not be their only judgment criterion, those who 
plan vocation education policies must have available to them data on 
the employment outcomes and experiences of those who have been trained. 
Because vocational programs are expanding and changing rapidly and 
because available data are spotty--both with regard to area and program 
coverage — it i c , necessary to develop comprehensive follow-up programs 
which can provide essential feedback. In addition to studies which 

3 

Lawrence J. Barnett, "Does Education for Work, Work?". The 
Urban Review . May 1966. 
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will provide systematic nationwide follow-up data for all levels of 
vocational education, special studies should be undertaken in the areas 
of the country which have been neglected, such as the South, and in 
program areas which often have been omitted or overlooked in research, 
such as postsecondary or supplementary training, home economics or 
technical education programs. The growing field of office occupations, 
which has only recently become a "vocational training" area in terms of 
federal involvement, should of course, be included in future studies. 
Because of the trend toward increasing employment of women, especially 
at the end of the child-rearing period, female as well as male graduates 
should be studied. The most promising current research studies— except 
in nursing — tend to be limited to men. 

For nationwide, comprehensive follow-up studies, there can be 
little doubt that the most productive follow-up design is a combination 
of trend and cohort studies. As shown earlier in this discussion this 
will yield the best kind of descriptive study, one that lends itself 
to meaningful analytical treatment. By matching vocational graduates 
in a given year with nonvocat ional graduates and follow-up of these 
graduates at different times one would be able to assess the differences 
due to training and one would be able to obtain a clearer picture of the 
effect of training over an extended period of time. At the same time, 
studies of new cohorts should be initiated at regular intervals, for 
example, every three years. In each case, random samples of men and 
women in all programs and at all levels should be included. An ideal 
scheme is outlined below (see sketch). A ten year program, involving 
between two and four separate contacts with three successive cohorts, 
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would yield not only the three cohort studies (of differing time span 
of course, ranging from three to nine years), but also several trend 
studies: those comparing cohorts, one and three years out of school 

at various points in time as well as people nine, six, and three years 
out at a given point in time (see sketch). While it may not be p act i cal 
to field so ambitious a study plan, the advantage of the successive 
cohort study should be taken into consideration in all phases of research 
planning. At the same time, for such studies to have explanatory power, 
it will be necessary to plan for simultaneous study of the other 
components of the “system" within which training and placement take 
place: the school and the labor market. Ideally again, in each of 

the contact years (1967, 1970, 1973, and 1976 in the example shown) 
institutional data (for example, class size, enrollment, teacher 
characteristics) and relevant labor market data should be collected 
and incorporated in the analysis. 

Concurrently with this elaborate set of successive cohort 
studies, the need for intensive small scale studies of particular areas, 
programs, or factors should be undertaken. The need for explanatory 
studies, far from being exhausted by the studies now in progress, 
becomes more pressing as new programs and techniques are being introduced, 
as they will be at an accelerated rate under the provisions of the 
Vocational Education Act. There is definite need to learn more about 
the role of specific institutional factors: type of school (comprehensive 

or vocational), size of school, role of teachers, curriculum experience 
and content, and guidance and placement services are but some of the 
areas where not enough systematic data is available so that judgments 
can be made regarding the types of variables to be used as inputs for the 

systems analysis referred to above. 
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1 LLUSTRATl VE SCHEMA FOR SUCCESSIVE COHORT STUDIES 



Contact Year 

Cohort A 
(June 1966) 

Graduates, dropouts, 
rejected applicants 
who would have 
graduated in this 
class. 



High School Graduation Year 



Cohort B 
(June 1969) 

Graduates, dropouts, 
rejected applicants 
etc. 



Cohort C 
(June 1972) 
Graduates, drop- 
outs, rejected 
applicants, etc 



June 1967 
June 1970 
June 1973 
June 1976 



1 year following 
i 

V 

3 year following 
6 year fol lowing 
9 year following 




1 year following 
3 year following 




6 year following 




1 year following 




3 year following 



The need for study is equally strong in the area of labor market 
requirements, employer preferences and behavior. Here too, many more 
explanatory microstudies must be deve1oped--wi th emphasis on sociological 
and psychological, as well as economic and technical factors — before 
sophisticated macrostudies can yield the kind of information which 
policy makers in the field of vocational education need. 

There is every reason to believe that with the initiation of 
comprehensive follow-up studies of the type recommended here, supplemented 
by intensive studies of particular factors and experiences in the 
training setting and the economy, a much clearer picture of the actual 
workings and effect ivness of vocational training would be obtained. 

This could be of immense help in bringing vocational education, 
occupational requirements, and productive individual careers into 
closer congruence. 
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to determine if high school vocational students are more likely 
to enter post-high school vocational programs after leaving 
high school than those who elect other types of high school 
programs. 

39. McKechnie, Graeme. "A Study of the Utilization of Vocational 

Education by Workers in LaCrosse, Wisconsin. " Annual Report . 
Madison: ' Center for Studies in Vocat ionol'and Technical Education, 
University of Wisconsin, 1965. 

A study, in progress, of vocational school students enrolled 
in night courses in terms of marital status, educational attain- 
ment, employment, and reasons for taking courses. Comparisons 
between students who completed training and dropouts were made 
and detailed analysis is planned for the future. 

40. Sewell, William H. "Factors in Educational and Occupational 

Aspiration and Achievement." Madison: Center for Studies in 
Vocational and Technical Education, University of Wisconsin, 

1965. (Proposal Summary.) 

An investigation of the social and social psychological factors 
associated with differences in educational aspirations and 
achievements of 10,000 Wisconsin high school seniors. Included 
is a follow-up of high ability students over a five year period 
with regard to college, entry into the job market, and career 
patterns. 

41. Twyman, J. Pascal. (Project Director) Progress Report - School 

Dropout Research Project . Stillwater: Research Foundation, 
Oklahoma State University, 1965. (Mimeographed.) 

A pretest postest design of a current study of an experimental 
vocational and academic program for recent high school dropouts. 
The dropouts have been divided into three experimental groups 
assigned to either vocational, academic, or vocational and 
academic training. Comparisons will be made with a control 
group which was not given any training. 
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42. Ulman, Lloyd. (Principal Investigator). “Entry and Job Change 

in a Large Metropolitan Labor Market Area." Berkeley: Insti- 

tute of Industrial Relations, University of California, 1966. 
(Mimeographed Study Proposal.) 

A proposal for a number of studies which will be concerned with 
the Ssn Francisco Bay area labor market and its relationship to 
vocational education needs. The following sub-projects are 
planned: analysis of labor market data for the purpose of pro- 

jecting occupational, industrial, and labor force changes; 
survey of employer policies and changes, survey of union policy 
and involvement in training programs, survey of coordination 
between employment and placement service personnel and school 
counsellors in providing vocational guidance by comparison of 
high school dropouts and graduates, study of labor mobility and 
vocational training, and study of the impact of automation and 
changes in skill requirements. 



III. METHODOLOGY AND BACKGROUND REFERENCES 



43. Craig, Judith S. (ed.). Conference on Follow-Up Studies in Educa- 
tional Research. Itovember. 1965 . Madison: Center for Studies in 
Vocational and Technical Education, Industrial Relations Insti- 
tute, University of Wisconsin, 19&5. 

Proceedings of a conference concerned with the objectives and 
methodology of follow-up surveys involved in vocational educa- 
tion evaluation. Included are participants' plans for follow-up 
research. 



44. Evans, Rupert N. (Chairman, Committee on Vocational, Technical, and 

Practical Arts Education). Review of Educational Research. XXXI I . 
4, Washington, D. C.: American Educational Research Association. 

Review issue devoted to vocational education includes descriptions 
of the types of vocational education as well as current research 
and a bibliography for each type. The chapter on distributive 
education by Harland Samson includes a discussion of the use of 
follow-up studies. 

45. O'Conner, Thomas J. Follow-Up Studies in Junior Colleges: A Tool 

for Institutional Improvement . Washington, D. C. American 
Association of Junior Colleges, l 965 . 

A discussion of the need for follow-up studies of junior college 
students and a manual which includes a detailed account of how 
to conduct a follow-up study. Sample questionnaires actually 
used by junior colleges in conducting their own studies are 
i ncluded. 
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46. Orr, David B. and Neyman, Clinton A., Jr. "Considerations, Costs, 

and Returns in a Large-Scale Follow-Up Study," The Journal of 
Educational Research . 58 (April, 1965). 

An article which discusses a follow-up study and concludes that 
a large-scale study is an expensive process and becomes costlier 
as non-respondents are urged to respond. In conducting the 
Project Talent follow-up study, authors found that a new question- 
naire was more effective in boosting the return rate than 
reminders. Efforts made to enlist the cooperation and involve- 
ment of students was believed to be worthwhile in terms of 
quantity and quality of responses. 

47. Schwarzwel ler , Harry K. Research Design, Field Work Procedure . 

and Data Collection Problems in a Follow-Up Study of Young Hen 
from Eastern Kentucky . (Rural Sociology Department. R. S. 21). 
Lexington: University of Kentucky, 1963. 

A discussion of the methodology used to conduct a follow-up 
study of male youth ten years after they were enrolled in the 
eighth grade. One aspect of this study planned to determine 
career patterns and economic life chances of the youth by 
securing data about first jobs, occupational mobility, levels 
of living, social mobility, career aspirations, and residential 
mobi 1 ity. 

48. United States Commission on Civil Rights Report - 3 (Employment) . 

Washington, D. C.: GPO, 1961. 

A report on the relationship between civil rights and employment. 
Contains some data on differences in vocational training and 
subsequent placement between whites and Negroes and on the role 
of unions. 

49. United States Department of' Labor. Nan-Power Report of the President 

and a Report on Manpower Requirements. Resources. Utilization, 
and Training , Washington, D. C.: GPO, 1964. 

A report to Congress of the most important developments and major 
continuing problems in employment and unemployment during 1963. 

. Areas of major concern, such as technological innovations, 

preparation for work, and disadvantaged workers are the subjects 
of separate chapters. Contains results of a special study showing 
where various categories of workers obtained training for work 
skills. 

50. Vivian, Neal E. "Evaluative Criteria for Distributive Education: 

Methods of Evaluation," Business Education Forum . 1ft (April, 

1961), Washington, D. C. United Business Education Association, 
National Educat ion Association. 

A short statement of methods of evaluation of distributive 
education programs and students taking these courses— test ing, 
use of student records, and on-the-job training evaluation. 
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IV. RELEVANT MATERIAL REQUESTED 
FOR THIS REPORT BUT NOT RECEIVED 



51. Larsen, Milton E. "A Study of the Characteristics of Students, 

Teachers and the Curriculum of Industrial-Technical Education 
in the Public Community College of Michigan." East Lansing: 
PhD Dissertation, Michigan State University. 

52. Nelson, Howard F. "Characteristics of Full-Time Students in 

Post-Secondary Trade Courses." Minneapolis: University of 

Minnesota, 1966. (Proposal). 

53. Smith, Max S. A Bibliography Relating to Selected Problems in 

Community Junior College Education . East Lansing: Michigan 

State University, 1961. 

54. Southern Colorado State College. "An Analysis of the i960 

Graduating Class at Pueblo College." Pueblo, Colorado. 



